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ABSTRACT 



This paper looks at the development and evaluation of an 



online learning and teaching package 

(http://www.herts.ac.uk/lis/ltdu/projects/mm2) in workshop techniques for 
Year 1 BA (Hons) Model Design students at the University of Hertfordshire 
(United Kingdom) . Students produce as part of their coursework physical 
models made from such materials as wood, metal, and acrylic. " Workshop 
Techniques" was designed to complement the series of practical course 
sessions that familiarize students with the workshop machinery. It 
incorporates an online resource and testing facility to back up the practical 
"hands-on" lectures in the workshops. Machinery is demonstrated and the 
students get a practical introduction to using the equipment. Students must 
attend this practical and complete the online tests to be judged "competent" 
to use the machinery on their own at a later date. This paper outlines 
problems that have arisen from the traditional method of teaching the course. 
It then discusses the author’s experimentation with a number of different 
teaching strategies, the first attempt which was to videotape the lectures. 
The next strategy was to present the material on the Web using a browser. The 
remainder of the paper discusses the design of the package, its applications, 
and the benefits of the feedback from student evaluations. (AEF) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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This paper will look at the Development and Evaluation of an On-line 
learning and teaching package in workshop techniques for students at the 
University of Hertfordshire. The package case studied, “Workshop 
Techniques”, is available on-line for viewing: - 

http://www.herts.ac.uk/lis/ltdu/projects/mm2 



This paper will address lessons that can be drawn from students 
working on-line using the “Workshop Techniques” package. It 
discusses 3000 test results and how student evaluations can improve 
teaching. 

I intend splitting this paper into three parts. The first will be an overview 
of the context of “Workshop Techniques”. The second on how the 
programme works from the student user point of view and the third on 
the evaluation process and how we as teaching professionals can learn 
from the students’ differing learning styles to feedback & improve our 
teaching methods. 
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Year 1 BA (Hons) Model Design students at the Faculty of Art & Design, 
University of Hertfordshire use "Workshop Techniques". This scheme, 
one of the premier schemes in Europe in this discipline, teaches students 
the practical skills necessary to gain employment in one of the following 
fields, product, architecture & special effects modelmaking. The students 
produce as part of their coursework physical models made from such 
material as wood, metal and acrylic. 

“Workshop Techniques” is an integral part of the first practical course the 
students participate in when they arrive. This course comprises a series of 
sessions to familiarize the students with the workshop machinery that they 
will have to be able to use to complete their projects later in the course. 
’’Workshop techniques” was designed to complement these practical 
sessions. It incorporates an On-line resource & testing facility to back up 
the practical “hands-on” lectures in the Workshops. Machinery is 
demonstrated and students get a practical taste of using the equipment. 
Students have to attend this practical and complete the on-line tests to be 
judged “competent” to use the machinery on their own at a later date. 




The traditional method in teaching students craft skills, particularly 
workshop and manufacturing processes in the field of Art & Design (such 
as Modelmaking) and related disciplines is one that is probably familiar to 
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all of you teaching practical subjects such as this. In the workshop skills 
sessions for example, they have followed a set pattern. For each machine 
in our workshop (there are currently about 60 different machines) students 
attend a lecture demonstration. A lecturer such as myself with technician 
support gives this. The students are divided into 3 or more small groups of 
about 16-18. The session starts by the Lecturer giving a demonstration of 
the safe working of the machine as well as describing its parts and their 
function. During this stage the students take notes from which to study 
later, these are useful if they do not operate that particular machine for 
some time after the lecture and aides their memory on what to do when 
they approach the machine at a later date. 

The second part of the session involves the students given a set 
task and in small groups, set up the machine and use it to 
produce an exercise piece each. For example, on a centre lathe 
each student turns up specific shapes using techniques learned 
during the lecture demonstration. Reinforcing their learning by 
doing it themselves at the end of the practical session each 
student thereby produces a simple artifact on the machine. They then 
leave and apart from a competence test which they have to pass, 
which is a written test on simple health and safety issues, they 
can use that machine on their own when they next come to the 
workshop to manufacture a model. 

The above two stages of a typical 1 . 5-hour session are pretty 
much how we have been teaching workshop skills for the last ten 
years. While being successful in initially imparting techniques 
for using machines, the format is not so successful in keeping 
the knowledge in the forefront of students minds, particularly 
when they may not use that particular machine for some months 
. After the initial lecture/demonstration session, the student then 
has to rely on the notes they have taken during the session. If 
they prove to be inadequate the only recourse they have is to ask 
for assistance in setting up from either one of the technical 
staff or the lecturer (usually me!) 

In addition problems always arise with the method outlined above 
in so much that in any large group of students, some will always 
miss the initial lecture. Some of the reasons why they do can be 
quite valid - such as being ill on the day. Sometimes the 
students are just absent. (Probably surfing the Net!). This 
however creates a problem for the University. To be covered in 
terms of insurance each student has to have received instructions 
on the safe operation of the machines. If they missed one lecture 
E.g. Milling machine. I have to repeat this lecture/demo session 
just for them. This is inherently wasteful in terms of 
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resources, giving one to one tuition to a student for an hour & a 
half is frowned upon in the current economic climate. 

Indeed doing this extra session for the student deprives the 
other 50+ students in that year group of my time whilst I am 
demonstrating. From the lecturers point of view this repeating of 
essential information to students who have missed it is boring, 

The teaching and non-teaching staff at the University have being 
trying for a considerable time to resolve this problem, of what 
to do with students who in the first case missed the introductory 
lecture. They could be given a "fast track" set up session by a 
member of the technical staff, this taking about 15 to 20 minutes 
immediately prior to the student starting work on the machine. 

However this is in no way as comprehensive as the taught session 
by the lecturer. It also runs into our perennial problem of 
whether Technical staff can be classed as "teaching" rather than 
demonstrating to students. This grey area can run into problems 
with role definition and job description. I attempted to find solutions to 
this problem , in order to give the students a better learning experience as 
well as addressing the staff time issue. 

To begin to tackle this problem I decided to experiment with a 
number of different teaching strategies. The first attempt being 
made by videoing the lectures . This was a partial solution, the 
lectures were taped from a single standpoint to begin with and 
the recorder was allowed to run live throughout the lecture. No 
change of viewpoint was used nor were there any cuts or close- 
ups. It was just a tape of the lecture. This was useful, as it 
allowed the student to have an overview of what they missed. 

However unless we spent a lot of money in terms of staff time 
setting up a "proper" TV style presentation of a particular 
machine, with close-ups , changes of camera angle and a specific 
narrative, it could not take the place of the initial lecture and 
the student still required the "fast-track set up" briefing 
before they started work. 

It was felt that taping the lectures was worthwhile in itself as 
it was found that when a student was asked to watch a video and 
take notes. They took in a fair amount of the information on the 
tape. This enabled us to ask the student to set up the machine 
when they wanted to use it and then check them, to see if what 
they had learned from the video was enough to safely prepare the 
machine for use. This was successful as the student could set up 
the machine without assistance using the knowledge gained from 
the video and only call staff over when they were ready to begin 
work. The time the member of staff was with them was reduced (by 




70%) as the student had completed most of the work themselves. 

This video trial was continued and each lecture was put onto 

tape. They are now in our library. There are 6 or 7, 3-hour tapes that the 

students can borrow or look at on site. The video system was not perfect 

however e.g. As each 1.5-2 hour lecture is recorded live on the tape and 

there are probably 5 or 6 lectures on different machines per 

tape. It was quite difficult for the viewer to fast forward 

through the whole lecture to find information on: for example, use 

of the dividing head in the Milling machine lecture. We tried 

indexing the tapes but as the viewpoint for each was constant 

throughout it was very difficult to glean information about a 

specific topic (within the lecture) without fast-forwarding 

through the lot. However it was still better than nothing as an 

aide memoir. I was still unhappy with the overall quality of the 

images (VHS - standard quality) and the fact that unless I 

wanted to commit a whole lot more time & money the viewpoint was 

fixed and no close ups were possible. I continued to look around for a 

better way of presenting the information. 

I also teach several sessions on computers, mainly for the special 
effects course where we use Adobe Photoshop to composite a model 
with a "real" background. The students are encouraged to use a 
variety of different packages in their work. I also noticed that students 
today are totally familiar with and use computers not only for their work 
at University but also as a hobby /relaxation tool. They surf the Net and 
use computer games like veterans. With this modern culture of hi-tech 
equipment & imagery evident in the students interest, I was interested to 
see if I could tap into this latent experience of Hi-tech gizmo's that the 
students are exposed to: For example the cinema & computer-games. How 
could this experience assist me in teaching a pretty dry and what almost 
could be said to be "traditional" technique of using woodwork and 
metalwork machinery to machine wood and metal to make physical 
artifacts. 

Although this is fun in itself it can look old and boring in the hi- 
technology world we live in. It is much more fun for the student 
of today to keep themselves clean, pushing buttons, this being in some 
strange way more "glamorous" than physically making something on a 
lathe. Where there could be a danger of them not only being hurt by an 
error on their part but heaven forbid even worse, getting their designer 
clothing contaminated with wood chippings, Oil and swarf! 

At that time I was writing pamphlets and course notes for the 
PhotoShop course & workshop skills machinery courses 
I was teaching, as well as the use of the World Wide Web for gathering 
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information. I hit on the idea of using the Netscape page and its ability to 
follow links and scroll down in a document, to make reading 
of documents simpler and more fun than a simple photocopied 
handout. 

By using Netscape I made the whole system of imparting 
information much more interesting and "Fun". I could also 
include Photographs in close up of the machine, showing 
particular operations as well as explanatory diagrams and 
illustrations. I put together some sample pages and put in a bid for funds 
from our Learning Technology Development Unit. (LTDU) I was 
successful in obtaining funding and proceeded to put together the finalized 
“Workshop Techniques” package. 

Presenting the material on the web using a browser was a stroke of genius. 
It tapped into the students' latent interest in anything remotely 
technological. Using the Internet and following links on a page is 
becoming a key skill as well, with 35% of adults regularly using the 
WWW not only for work but also for recreation enjoyment. 

Not only were the pages indexed so those students could refer back 
to what they were looking at previously, but also it was much 
clearer then scrolling through an entire videotape. The pages were 
divided into sections for ease of use with a page on each machine 
as well as linked pages on e.g. Operation, Health and Safety tips 
Etc. 

Students familiar with the keyboard skills and using the Net to 
"surf for information as well as a recreational facility can use the net to 
surf through the information in the package. By following the links they 
can also take in as much information as I can give them. In the case of the 
student who has totally missed the lecture they would have to follow all 
the links telling them all about the machine and its attachments. A student 
however who had attended the initial lecture who only wanted 
information on one particular aspect could skip through most of 
the pages and links to gain the specific information that they 
wanted. 

Once I had secured funding to proceed with the project I started work in 
the summer of 1998. The first thing I realized was the enormous amount 
of work I had undertaken to do, in terms of generating the text and in 
addition posing and taking over 450 images of various machines. The 
package was originally put together using Dreamweaver 1.0. This resulted 
in a non-frames version. It was divided up into sections, each 
section covered setting up and using each machine. The emphasis being 
placed on health & safety issues, with separate sections dealing with safety 
equipment such as masks and goggles. I made the decision to incorporate 
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on-line testing to reinforce what the students had learned in reading the 
initial “Tutorial” part of the package. As the key aim of the package was to 
ensure that students who had missed the initial lecture had remembered 
sufficient information from the “tutorials” section so that they could this 
information to good use when they came down to the workshop to use a 
machine for the first time. 

After some initial development using the non-frames version it was 
decided that due to the learning content covering the usage of ten 
machines and the high density of images, a frameset structure was 
adopted. This allowed me to conceive a structure whereby the 
navigation buttons (e.g. options to choose machine type, test section 
etc.) would be permanently on display to the user and be accessed at 
any point throughout the material - even during the test section. Thus the 
nature of the project altered from one in which the test sections were to 
be placed and accessed at specific points within the material to one 
where the questions could be accessed freely. 

This frame based approach provided an “open” learning environment 
and suited the requirements of the project in that the students were able 
to test their current knowledge of the subject whenever they wished and 
thus gained some sense of their progress. 

At the outset of the project it wasn’t totally clear what software would be 
used to create the assessment sections. The public domain package Web 
Course in a Box was tested and seemed to be adequate to the task but 
only seemed capable of multiple choice single answer questions, or fill 
in the blanks sentences. The Castle project was also inspected, but this, 
like Web Course in a Box, had most of the marking undertaken at the 
server side. As the demand had arisen for distribution by CD-ROM as 
well as over the web, specifically for colleagues in the USA, we needed 
something, which would also function on the client side alone if 
possible. 

Moreover, we wanted something whose look and feel would be entirely 
controllable by other developers and myself in order that its questions 
could be seamlessly embedded in a teaching package. In the end 
therefore, LTDU developed its own question authoring system, the 
Question Generator which could produce the complete test in the form of 
a single html file. Results from students sitting the test were sent to 
me as emails via a CGI script on the University of Hertfordshire’s 
web server: however, even if the person using the system had no web 
connection and used a CD-ROM, the quizzes could still be used for self 
assessment purposes. 
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The Question Generator works by the author writing the questions on a 
web form, following a simple syntax which are then posted to a cgi- 
script which returns an HTML page with the quiz encoded within it in 
javascript. Once this was done, we would then incorporate the 
resulting quizzes into a frameset template using Macromedia 
Dreamweaver. The final test page would be nested as a secondary 
frameset within a parent frameset, (which displays the navigation 
buttons). A typical test section took approximately 1 hour to produce. 

A general point to be made here is that flexibility-of-deployment needs to 
be a very important consideration when using any kind of computerised 
assessment. The number of possible contexts in which one may wish to 
which is not too exigent in terms of look and feel, and precludes as little 
as possible in the areas of customisation and modifiability is to be 
preferred. 

Each test section produces an overall score and a summary of the results 
for each question. On the basis of the scores the student can decide 
whether they are able to move on to study another machine or should re- 
visit certain sections. The score is used as a means of determining the 
students' direction of learning through the package. In this way 
Workshop Techniques is responding to the students misconceptions of 
the topic and redressing them by indicating the areas of the topic which 
the student is weak in and should re-consider. The use of diagnostic 
testing for directive study enables a class of students to come to a 
common learning point within the course material regardless of either 
their ability or prior knowledge of the subject. 

After completing the test the student would submit their answer to the 
program for scoring. At the same time a total score along with the time at 
which the test was taken is forwarded by email to me. This serves 
a number of purposes: first I can actually see if the package is being 
used at all; second I can gain an idea which students are actually using 
the program and who are not and thirdly they can monitor the score for 
the whole class. If I find that the class appears to be struggling I 
can arrange for extra tuition in the form of revision lectures and 
practicals. 

The use of embedded testing within Workshop Techniques lends itself 
to three types of usage: 

First the embedded test along with the rest of the application forms part 
of an integrated course where Workshop Techniques is supported by the 
more traditional lecture and practical. In this context the application acts 
the concepts introduced within the lecture. 
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Second the application is used as a means of determining the gaps within 
the knowledge of students who have either missed the practical / lecture 
sessions or who are struggling with the course content. From the student’s 
test scores I can see in what areas of the course the student is having 
problems with and determine whether the student does indeed have to 
attend a practical and if so the length and extent of the session. In this 
context I am able to determine the degree of traditional support 
appropriate to the student’s needs. The presence of overseas students 
within the course has been steadily increasing over the last few years with 
many individuals seeking placement beyond the first year. Under these 
circumstances Workshop Techniques is used within an “interview” 
context whereby the eligibility of the student to enter either the second or 
third year of the course is ascertained. On the basis of the test scores the 
staff can decide whether the student can be moved up and if so what type 
of additional support they may require. 

At this point I would like to point out that all these tests are not looked 
upon by me as being in the same league as a formal examination, with all 
the issues such as security to be addressed. As the entire point of the 
“Workshop Techniques” is to concentrate on imparting health & safety 
information, I am quite happy if the students keep referring back to the 
tutorials whilst doing the test as well as asking each other questions whilst 
sitting together on workstations. In any event I have yet to come across a 
system where the teaching staff are absolutely sure that submitted work in 
any format e.g. essay has been written by the student submitting it, or 
come to that downloaded it off the internet! 

Now that the “Workshop Techniques” package has been up and running 
since 1998. 1 turned my attention to evaluating its performance and how 
the students' evaluation of it as a teaching method can be used to affect the 
future shape of this and other web based methods of instruction. 

Evaluation of “Workshop Techniques” has been gleaned from principal 
sources. First was the actual e-mail test results sent to my PC. This gave a 
list of the questions answered but also how many tries the student had 
taken to get the answer correct. These e-mails also recorded the time of 
day the student hit the “submit” button to send the test on. As an 
additional by product of the test. The student had to fill in their name and 
e-mail address in the relevant boxes-this meant that immediately I had a 
full list of all the participating students' e-mail addresses. A task that can 
take many hours if requested orally in the class environment! 

The second source of information was from the Universities own web 
server. This gave information on how many hits each page had received 
and at what time of day. From this you could deduce which pages were the 
most popular and also which order they were accessed. You could also 




find out if the students referred back to the relevant pages of the tutorial 
sections whilst looking at particular tests. In essence, this allowed 
individual students' progress through the package to be mapped if 
required. 

The final source of data was an in-built evaluation questionnaire built into 
the package as an integral link. This is under the link “feedback”. This has 
been used by the students as well as outside contacts as a form of peer- 
review. It covers such areas as what type of computer and browser the user 
is using to access it. As well as specific questions about the way in which 
a particular student accessed the information. Importantly it asked the 
student how familiar they were with navigating through teaching packages 
such as this, as well as their level of computer literacy. 

Because use of the package was an in-built part of a course, take-up by 
students was very high. Some 90% of the class used the package, 
particularly the tests. Use of the tests has resulted in the generation of over 
3,000 e-mail responses to date. What was unexpected was the amount of 
time each individual student spent on using the package. Figures of 7 to 8 
hours were not uncommon. The number of hits the pages got at different 
times of day was also interesting. The largest number of hits was recorded 
between midnight and lam and Sunday afternoons between about 2.30pm 
and 4 O'clock. This was surprising as prior to the package being developed 
students only attended a 1.5-hour lecture/demonstration and did a simple 
multiple choice test based on a slide presentation. 

One of the most encouraging things when developing on-line applications 
is this kind of tangible feedback one can get about one's teaching and 
materials. When you see all the e-mail coming in, detailing one's students' 
understanding of the materials, as well as more interesting findings of the 
kind described above, it rewards all the effort you put in. In this particular 
case, the instant feedback from tests had many uses: you could 
immediately see whether a student was competent to be able to use the 
machines on their own. This was valuable for the faculty (as mentioned 
previously) because Health & Safety requirements demanded that each 
student was deemed to be competent to use the machinery. Instead of 
long- winded tests in a lecture setting (which had to be physically marked) 
the package instantly told me the percentage each student got right on each 
test straightaway. So, there was no lead-time between the student taking 
the test and being documented as having passed (or failed) the test. 

Initial findings have also discovered a range of learning styles employed 
by the students in how they navigated around the information provided. 
Different learning styles have become apparent. These differences in 
approach have in turn fed back into the development of the package in 
terms of navigational references and other link based paths. These 
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differences in learning style have included students who started the project 
and followed it through methodically from start to finish, Others used a 
much more pick and mix approach, repeating some sections, and ignoring 
others. The on-line assessment was also accessed directly by other 
students who only used the tutorial sections after they had completed the 
tests. The most significant was the fact some students learning style 
involved accessing the package on a 24 hour learning basis. A point that I 
had not anticipated at first when I first came up with the idea of 
“Workshop Techniques”. I have had examples of students submitting test 
results between 3 and 4am! 



Ashley Pinn July 2000 
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